What is this about?

This is brainstorming on how to extend paramb in order to have full control over the layout of the underlying widgets. The implementation should meet different objectives:

  1. Decouple the logic of a python class from its dashboard representation:

    1. The actual functionality of the class should follow the paramnb filosophy.
    2. There should be a way to reference the widget representation of every parameter.
  2. Offer a way of quickly define complex combinations of parameters and interactivity options.

  3. Capability to combine different classes representing dashboards. This would allow to create more complex dashboards through object inheritance and composition.

1. The paramnb way

In paramnb the parameters are defined at class level following the following sintax:


In [115]:
import param
import paramnb
def hello(x):
    print("Hello %s" % x)
    
class BaseClass(param.Parameterized):
    x                       = param.Parameter(default=3.14,doc="X position")
    y                       = param.Parameter(default="Not editable",constant=True)
    string_value            = param.String(default="str",doc="A string")
    num_int                 = param.Integer(50000,bounds=(-200,100000))
    unbounded_int           = param.Integer(23)
    float_with_hard_bounds  = param.Number(8.2,bounds=(7.5,10))
    float_with_soft_bounds  = param.Number(0.5,bounds=(0,5),softbounds=(0,2))
    unbounded_float         = param.Number(30.01)
    hidden_parameter        = param.Number(2.718,precedence=-1)
    
class Example(BaseClass):
    """An example Parameterized class"""
    boolean                 = param.Boolean(True, doc="A sample Boolean parameter")
    select_string           = param.ObjectSelector(default="yellow",objects=["red","yellow","green"])
    select_fn               = param.ObjectSelector(default=list,objects=[list,set,dict])
    int_list                = param.ListSelector(default=[3,5], objects=[1,3,5,7,9],precedence=0.5)
    single_file             = param.FileSelector(path='../*/*.py*',precedence=0.5)
    multiple_files          = param.MultiFileSelector(path='../*/*.py?',precedence=0.5)
    msg                     = param.Action(hello, doc="""Print a message.""",precedence=0.7)

In [117]:
paramnb.Widgets(BaseClass())


1.1 Advantadges

  • Do not pollute the domain-specific code: Same domain speciffic code, widgets for free.
  • Great inheritance mechanism.
  • Layout auto-tuned using the precedence parameter.
  • Clean way to override default parameters.

1.2 Cons

  • Impossible to access widget specific parameters/tune widget interactivity.
  • No easy way to implement display-related logic.
  • Impossible to define the widget layout manually (only using the jupyter dashboards extension).
  • No easy way to propagate callbacks in complex dashboards.

2. The shaolin way

Shaolin tries to follow the same filosophy as paramnb while providing full control over how the user will interact with the widget GUI, at the cost of some pollution in the domain-specific code.

This is a list of cool stuf that shaolin can do. It would be great to find a way to include these features in paramnb.

2.1 accessing widget specific parameters/tune widget interactivity.

Shaolin transforms every parameter of a class into a shaolin widget, wich is a wrapper for an ipywidgets Widget.


In [2]:
from shaolin import shaoscript
shao_param = shaoscript('togs$description=miau&options=["paramnb","shaolin"]')#let's create a shaolin widget
shao_param,shao_param.value,shao_param.widget,shao_param.name


Out[2]:
(<shaolin.core.widgets.Widget at 0x7f1d8d67ee10>,
 'paramnb',
 <ipywidgets.widgets.widget_selection.ToggleButtons at 0x7f1d8d67eda0>,
 'miau')

This mean that domain spacific code will be poluted because in order to access the value of the parameter we have to access the value attribute of the shaolin widget or call the shaolin widget. The polution will be either adding a () after each parameter, or wirting shao_param.value.


In [3]:
shao_param(),shao_param.value


Out[3]:
('paramnb', 'paramnb')

The ipywidgets representation of the parameter can be accessed using the widget attribute or indexing the shaolin widget with an integer.


In [4]:
shao_param.widget



In [5]:
shao_param[0]#indexing with any integer shows the widget representation


2.2 multiple callbacks in the same class

It is important to be able to assign several callbacks to the same class. Generally speaking, when programing a dashboard two types of callback will be needed:

  • Domain speciffic callback: contains the code of the function that will use the widgets as parameters.
  • Layout callback: Logic that controls how the dashboard is displayed.

In the following example we have an example class with 4 different callbacks applied to its widgets.


In [9]:
from shaolin import Dashboard
from IPython.core.display import clear_output
class ShaoExample(Dashboard):
    
    def __init__(self):
        dashboard = ['column$name=example',
                     ['toggle_buttons$description=Toggle&name=toggle&options=["paramnb","shaolin"]',
                      'textarea$description=Some text&name=text&value=brainstorming',
                      'float_slider$description=Float slider&name=fslider&min=0&max=10&step=1&value=5'
                     ],
                    ]
        Dashboard.__init__(self,dashboard)
        self.fslider.observe(self._update_layout_1)
        self.fslider.observe(self.callback_2)
        self.toggle.observe(self.callback_1)
        self.toggle.observe(self._update_layout_2)
        
        
    def _update_layout_1(self,_=None):
        """Hides the textarea widget when
        the slider value is higher than 5"""
        if self.fslider.value >=5:
            self.text.visible = False
        else:
            self.text.visible = True
            
    def _update_layout_2(self,_=None):
        """Disable the text input when the
        toggle buttons value is shaolin"""
        if self.toggle.value =='shaolin':
            self.text.widget.disabled = False
        else:
            self.text.widget.disabled = True
    
    def callback_1(self,_=None):
        """Prints the textarea value"""
        print(self.text.value)
        clear_output(True)
    
    def callback_2(self,event):
        """Displays the event data passed
        to the callback and the slider value"""
        print("event data: {}".format(event))
        print(self.fslider.value)
        clear_output(True)

In [7]:
shao_e = ShaoExample()

In [8]:
shao_e[0]


2.3 Accessing the dashboard kwargs

Instead of printing the parameters, it comes really in handy to hace a dictionary with all the dashboard parameters. This allows you to make a dashboard with no domain-specific callback, so you can use it as a way to organise parameters.


In [149]:
shao_e.kwargs,shao_e()


Out[149]:
({'fslider': 5.0, 'text': 'brainstorming', 'toggle': 'paramnb'},
 {'fslider': 5.0, 'text': 'brainstorming', 'toggle': 'paramnb'})

If the only thing needed is a graphical interface capable of providing a kwargs dictionary managed by widgets I usually use the KungFu class.

2.4 Displaying the dashboards

When creating a complex dashboard it is nice to have two diferent ways of organising the layout:

  • Programatically.
  • Using the jupyter dashboards extension.

In order to define the layout programatically shaolin uses syntax based on a list of lists that mimics how the flex-layout of the ipywidgets package is defined.

The convention is the following:

[box_widget, [children_1, children_2]]

where children can also be a list of list with the structure defined above.


In [102]:
import seaborn as sns
%matplotlib inline
sns.set(style="ticks")
data = sns.load_dataset("anscombe")

title = '#Exploratory plots$N=title&D='
marginals = "['Both','None','Histogram','KDE']$d=Marginals"
dset = "['ALL','I','II','III','IV']$D=Dataset"
x_cols = 'dd$D=X column&o='+str(data.columns.values.tolist())
y_cols = 'dd$D=Y column&o='+str(data.columns.values.tolist())+'&v='+data.columns[1]
save = "[False]$D=Save plot&n=save"
data_layout = ['c$N=data_layout',[title,['r$N=sub_row',[x_cols,y_cols]],
                                  ["c$N=sub_col",[marginals,dset,
                                                  ['r$N=btn_row',['@btn$d=run&button_style="info"',save]]                                                 ]
                                  ]]
              ]
dash = Dashboard(data_layout,name='dash_1',mode='interactive')
dash[0]


Children is defined as string following the shaolin syntax. This string representation allows to quickly set the parameters and interactivity of every widget and act as a proxy for defining shaolin widgets.

This way we are able to define how we want our dashboard to be displayed. Once a dashboard is created, it is also possible to alter how it will be displayed using the jupyter dashboards extension.

In oder to do that, you can render each component of the dashboard in a different cell, and then rearange the cell using the dashboards extension:


In [103]:
dash.title[0]



In [104]:
dash.sub_col[0]



In [105]:
dash.sub_row[0]


2.5 combining different dashboards

It is also really useful to be able to combine different dashboards. Shaolin allows Children to be a shaolin dashboard as it is shown in the following example:


In [106]:
#regression
r_title = '#Regression options$N=r_title&D='
reg = '@[True]$D=Plot Regression&n=regression'
robust = '@False$D=Robust'
reg_order = "@(1,10,1,1)$D=Reg order"
ci = "@(1,100,1,95)$N=ci&d=Confidence intervals"
reg_layout = ['c$N=reg_layout',[r_title,reg,reg_order,ci,robust]]
dash_2 = Dashboard(reg_layout,name='dash_2',mode="interactive")
dash_2[0]



In [107]:
combined_dashboard = ['row$n=combined',[dash,dash_2]]
comb_dash = Dashboard(combined_dashboard,mode='interactive')
comb_dash[0]



In [108]:
comb_dash.dash_1[0]



In [109]:
comb_dash.dash_2[0]


2.6 propagating callbacks to children

In shaolin there are three diferents types of interactivity modes that a parameter can have:

  • Interactive (@): Interactive widgets will be assigned the target callback function when dashboard.observe(callback) is called. Interactive parameters will be included in the kwargs dictionary.

  • Active: Default mode. Active widgets wont get any callback applied when dashboard.observe is called but they will appear in the kwargs dictionary.

  • Passive (/): Passive widgets wont be included in the kwargs dictionary and don't get callbacks automatically applied.

In order to apply a callback to an active/interactive widget, target_widget.observe(callback) must be splicitly called.

In the following example the callback function will be propagated to any interactive widget (its definition starts with a "@"). This means that dash_1 will execute the callback when the run button is pressed and when any widget of dash_2 is used.


In [110]:
def callback(event):
    print(event)
    clear_output(True)
    #print(event,other)
comb_dash.observe(callback)
comb_dash[0]


{'owner': <ipywidgets.widgets.widget_int.IntSlider object at 0x7f1d8d2832b0>, 'name': 'value', 'type': 'change', 'new': 1, 'old': 2}

3. Brainstorming: How to add this features to paramnb

It would be nice to find a way to implement these features without breaking the paramnb filosophy. I dont know which one is the best way to do it, but here are some ideas:

Accessing the widgets

Mabe it would be usefull to create aditional attributes of a class containing the widget representation of a variable.

For example, in the Example class it would be great to access the example.boolean (True) value this way, and the checkbox widget could be accessed like this: example._boolean (CheckBox).

Another option would be:

  • example.boolean (True)
  • example.layout.boolean (CheckBox)

It would be great to find a good naming convention.

widget definition

The shaolin string syntax used may look really messy and complex, but once you get used to use it it literally saves hundreds of lines of code and makes it really easy to mantain the layout and make changes with some clever copy-paste.

It would be great to standarize some sort of pseudo programing language for creating layouts with the ipywidgets package. This way, the BaseClass from the following example could also be defined the following way:


In [148]:
class BaseClass(param.Parameterized):
    x                       = param.Parameter(default=3.14,doc="X position")
    y                       = param.Parameter(default="Not editable",constant=True)
    string_value            = param.String(default="str",doc="A string")
    num_int                 = param.Integer(50000,bounds=(-200,100000))
    unbounded_int           = param.Integer(23)
    float_with_hard_bounds  = param.Number(8.2,bounds=(7.5,10))
    float_with_soft_bounds  = param.Number(0.5,bounds=(0,5),softbounds=(0,2))
    unbounded_float         = param.Number(30.01)
    hidden_parameter        = param.Number(2.718,precedence=-1)

class BaseClass_(param.Parameterized):
    #custom syntax highlighting using pygments would be really awesome. Red is hard to read
    layout = ['r$N=baseclass',[['c$N=text_col',['ft$d=X&v=3.14&doc="X position"',
                                            'ft$d=Unbounded float&v=30.01',
                                            'it$d=Unbounded int&v=23',
                                            'text$d=String value&v=str&doc="A string"']
                               ],
                               ['c$N=sliders_col',['(7.5,10,0.1,8.2)$D=Float with hard bounds',
                                                   '(0.,2.,0.1,0.5)$D=Float with soft bounds&hb=(0.,5.)',
                                                   '(-200,100000,1,50000)$d=Num int&v=23',
                                                   'text$d=y&v=Not editable&disabled=True']
                               ]
                              ]
             ]

In [124]:
paramnb.Widgets(BaseClass)


this is how it looks


In [146]:
layout = ['r$N=baseclass',[['c$N=text_col',['ft$d=X&v=3.14&doc="X position"',
                                        'ft$d=Unbounded float&v=30.01',
                                        'it$d=Unbounded int&v=23',
                                        'text$d=String value&v=str&doc="A string"']
                           ],
                           ['c$N=sliders_col',['(7.5,10,0.1,8.2)$D=Float with hard bounds',
                                               '(0.,2.,0.1,0.5)$D=Float with soft bounds&hb=(0.,5.)',
                                               '(-200,100000,1,50000)$d=Num int&v=23',
                                               'text$d=y&v=Not editable&disabled=True']
                           ]
                          ]
         ]
wannabe_dash = Dashboard(layout)
wannabe_dash[0]



In [147]:
wannabe_dash()


Out[147]:
{'float_with_hard_bounds': 8.2,
 'float_with_soft_bounds': 0.5,
 'num_int': 50000,
 'string_value': 'str',
 'unbounded_float': 30.01,
 'unbounded_int': 23,
 'x': 3.14,
 'y': 'Not editable'}

In [ ]: